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Faculty at Utah State University sought to modify a 
curriculum for teaching professionals the skills of naturalistic intervention 
with preschool children with disabilities to make it suitable for primary 
caregivers, and then to offer the modified curriculum over the Internet to 
maximize caregiver access. The curriculum development team decided to use 
Internet-based focus groups to obtain consumer input because of rural 
location, caregiver difficulties attending meetings, and the goal of national 
dissemination. Due to the lack of research on the efficacy of Internet-based 
focus groups, the team decided to begin the focus group in a face-to-face 
manner and then transition the group to Internet-based sessions. Six parents 
of children with disabilities participated in the focus group, providing 
feedback on the Web site, the curriculum, and the information offered. Two 
major' modifications resulted from focus-group recommendations: using primary 
colors to indicate the Web site was child-related, and providing parents with 
video examples on CD-ROM to minimize download time. Findings indicate that 
focus groups can be effective in either format, but an Internet-based focus 
group has some interaction strategies that differ from those employed in 
face-to-face sessions. Recommendations are offered for setting up and 
facilitating online focus groups. (TD) 
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Increasingly, and for good reason, service delivery programs are seeking consumer input during their 
planning and development stages. In the past “experts” developed services and delivered them as pre-packaged to 
consumers under the assumption that experts knew best how to create curriculum and other services. If a service did 
not meet consumers’ needs, it was a problem with the consumer, not with the experts or the delivered service. 

Often, the services were appropriate and meaningful to consumers. When services were not appropriate and 
meaningful, however, consumers were often captive audiences restricted by time, space, and knowledge from 
critiquing the program or from seeking alternatives that more directly met their needs (Rowntree, 2000). In best 
case scenarios either consumers were able to modify their learning strategies and make best use of the service or the 
experts developed new versions of their programs that, hopefully, better met consumers’ needs. Consumers of 
special education services were no different in this respect, frequently accepting or working around programs that 
only marginally met their needs. 

More recently, as recognition of consumers’ expertise has increased, the incorporation of consumer input in 
service development has increased as well. Now, consumer participation is considered important from the initial 
stages of program and service development, to the completion of the final product, to the development of revisions 
and alternatives that more closely met consumer needs. In this manner, service delivery has become more consumer 
driven, ensuring that services meet the needs of their intended audiences. The question becomes, at this point, how 
to gain effective consumer participation in the process of program and service development. For special educators, 
consumers include family members and the children themselves— certainly those at least as young as elementary age. 
Focus Groups are one Means of Achieving Consumer Input 

Focus Groups are one means of identifying consumer needs and preferences (Merton, Fiske, & Kendall, 
1990). They allow potential consumers to examine a service or a proposed service and provide immediate feedback 
that can then be used to more effectively develop and deliver that service or product. Traditionally, focus groups are 
based in the use of face-to-face group discussions. Such group discussions are easily facilitated in large population 
clusters where members of the target population are plentiful. In comparison, focus groups are difficult to facilitate 
in rural areas where members of the target population are few and are spread over large areas. Due to difficulties 
related to the long distances it is necessary for participants to travel and to the scarcity of potential participants, the 
use of face-to-face focus groups in rural settings is quite problematic, if not impossible. 

Potential of the Internet to Increase Focus Group Participation 

The Internet, with its capability for facilitating communication between individuals separated by space and 
time, has greatly expanded our notions of how people can communicate and under what conditions they can develop 
working relationships. Internet-based communication tools such as chat rooms, emails, threaded discussions, and 
listservs all provide the opportunity for individuals separated by great distances to join together an<^ share 
experiences, opinions, and ideas. 

Given the options for Internet-based group interactions, it seems plausible that focus groups could be 
conducted via the Internet. While several articles exist that advocate the possibilities of Internet-based focus groups 
(e.g., Chepesiuk, 1996) research that evaluates the efficacy of Internet-based focus groups is minimal (Adriaenssens 
& Cadman, 1999; Fulop, Loop-Bartick, & Rossett, 1997; Tse, 1999; Wheller, 1996). Additionally, some focus 
group experts have expressed the belief focus groups cannot be effectively conducted over the Internet. Greenbaum 
(1998) for example, states that the communication limitations of the internet (lack of face to face interaction, lack of 
nonverbal communication, and inability of the moderator to personally interact with focus group members) make it 
impossible for focus groups to exist in an internet environment. As previously noted, however, the limited research 
base is insufficient to evaluate the merit of these concerns. 
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Necessity is the Mother of Innovation 



The SPIES Outreach Project began to consider the possibility of internet-based focus groups when we saw 
the need to obtain consumer input into development of a Web-based version of the curriculum. Strategies for 
Preschool Interventions in Everyday Settings (SPIES) is a validated curriculum for personnel preparation of 
professionals and paraprofessionals to teach the skills of naturalistic intervention with preschool children who have 
or are at risk for disabilities. In its current format, SPIES is a six module, videotape and manual based curriculum 
funded by a multiyear grar* from the Office of Special Education Projects (OSEP). An abbreviated version is 
available on CDROM. One of our objectives for the project is to modify SPIES to make it suitable for primary 
caregivers of children with disabilities and to then offer the modified curriculum over the internet as a means of 
maximizing' the opportunities for caregivers to access it. 

Operating out of the Center for Persons with Disabilities (CPD) at Utah State University, we were faced 
with several difficulties in securing consumer participation through traditional means. First, Logan, Utah, where 
CPD is based, is a predominately rural region. Second, it is frequently difficult for caregivers of preschool children 
with disabilities to make the time commitments and arrange for appropriate child care necessary to participate in the 
several sessions we believed would be necessary to gain adequate consumer input. Concurrently, because SPIES 
has a national dissemination goal, we wanted to develop a system that would enable us to obtain consumer input 
from across the country. For these reasons, we began to evaluate the possibility of attempting to utilize the Internet 
as a means of assembling prospective focus group members into a viable forum. In the process of this evaluation, 
we faced the lack of research into this area, but rather than viewing this as an impediment, we saw an opportunity to 
both meet our needs and to gain information into the efficacy of internet-based focus groups. 

Recognizing, however, that we were entering relatively uncharted territory, we decided to begin our focus 
group in the traditional face-to-face manner and then transition the group to internet-based sessions. This also 
provided us the opportunity to evaluate similarities and differences between the two modalities — face-to-face versus 
internet-based. We thus sought to answer three questions. First, how should we modify the SPIES curriculum to 
make it appropriate for parents and could we make it deliverable over the Internet? This question was essentially 
independent of whether the focus group was conducted in person or over the Internet. Second, can effective 
consumer input be obtained when a focus group is conducted over the Internet. Third, are there differences in focus 
group effectiveness and interaction when conducted in person or over the Internet. 

Methods 

Participants 

Participants were six adult parents of children with disabilities. Among the six participants, there were two 
married couples. A seventh adult (husband of one participant) participated in the last focus group. The disabilities 
experienced by the children ranged in severity from massive developmental delay with multiple sensory 
impairments to hearing loss with ear deformation. All participants lived in the Cache Valley of northern Utah. 

Their residences ranged from living in the largest community of the valley (approximately 46,000) to the smaller 
outlying towns. Participants were solicited through advertising in the valley-wide newspaper, through the local 
parents’ support group, through a preschool preparation program, and through an early intervention program. 
Participants received honorariums in the amount of $40 per session attended. 

Organization of Sessions 

Four focus group sessions were conducted from September 19, 2000, through December 12, 2000. The 
first two groups were conducted approximately 2 weeks apart. We then paused for approximately one month in 
order to build a Website to the specifications identified in the first two groups. The last two sessions were 
conducted six weeks apart due to scheduling difficulties related to the holidays. 

Face-to-face sessions. 



The two face-to-face sessions were conducted according to standard focus group principles (Vaughn, 
Schumm, & Sinagub, 1996). Because the sessions were conducted in the evening, food and beverages were 
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provided. Two weeks prior to the first session, participants received through the mail a CDROM version of the 
SPIES curriculum and its corresponding manual. The CDROM contains a well-evaluated, abridged version of 
SPIES. Participants were asked to view the CDROM as if it were a Website. Because the goal of the first session 
was to elicit general impressions and suggestions, participants were requested prior to the first session to “play” with 
the CDROM rather than attempting to formally work their way through the curriculum. For the second session, 
participants were requested to formally progress through the CDROM in preparation for discussing specifics of the 
curriculum as well as navigation and “look and feel” issues that might guide the development of the Website. An 
agenda of questions available from the fir:*, author was followed for each session. The first and second face-to-face 
sessions were intended to be recorded on microcassette and then transcribed. Due to error on the part of the 
moderator, the first session was not recorded. A synopsis of this session was created by the moderator and delivered 
to each participant by email with the request that they edit or add to the synopsis as necessary to capture their 
participation in the session. The second session was successfully recorded and transcribed. After transcription, the 
microcassettes were erased. 

Internet-based sessions. 



The two Internet-based sessions were conducted via a proprietary chat program provided by the Acropolis 
Online Learning Environment at the University of Utah Center for Persons with Disabilities (The Acropolis 
Chatroom is designed to be accessible to individuals with disabilities). In both Internet sessions, an agenda of 
questions developed on the basis of prior session responses and on the basis of developments of the Website were 
followed. Because nonverbal communications were unobservable, group members participated in drafting a set of 
protocols for determining when to proceed to the next scripted question. Similarly to face-to-face sessions, 
flexibility in questioning was maintained in order to address issues as they arose during the sessions. The Internet 
sessions focused on the design of the Website both generally and specifically. Generally, participants provided 
feedback on issues such as ease of access, look and feel, and movement within the site. Specifically, they provided 
feedback on specific issues such as colors, graphics, videos, and placement of items. They also provided feedback 
on the quality of the curriculum and appropriateness of the information presented. All communications in the 
Internet-based sessions were recorded as a function of the Acropolis Chat Program. These verbatim transcripts were 
downloaded onto a password protected computer system and then erased from the chat program. 

Data analysis. 



Using the transcripts, responses were reviewed to identify major themes present in each session. In this 
manner, five major themes were identified and are listed in Table 1 . All responses were then categorized into one of 
the identified themes. Once categorized, responses were reviewed for content relevant to each of the research 
questions. 



Table 1 

Major Themes and their Subthemes 
Theme Subtheme 

1. Informal Relationship Building 

Information Sharing and Seeking 

2. Presentation Movement 

Look 

3. Information Usefulness 

Accessibility 

4. Group Facilitation 

5. Overall Impression 



Results 

Curriculum Modifications: 

Our first research question related directly to the primary objective of developing an effective, accessible, 
attractive, Website. We began the internet-based focus groups with a prototype Website that group members 
interacted prior to providing feedback about it. This prototype Website was built according to current best practice 
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standards in web-based design. Group participants, however, were negative in their responses to the “look and feel” 
of this site. They found it too cold and professional. They expressed the opinion that a caregiver-oriented site 
should employ primary colors and should immediately give some indication that the site was child related. In 
response to this feedback, our second prototype made extensive use of primary colors and child appropriate pictures 
(toys and animals). Response from participants about this site indicated that we had moved too far in this direction. 
Their feedback indicated that they found the site too cute and child-like giving them the impression that there would 
be little information of substance on the site. Thus, our final version of the Website maintained it’s emphasis on 
primary colors but did not employ child appropriate p’Ctures. 

A second area of significant importance in site development focused on video examples. The SPIES 
curriculum relies heavily on video examples to provide models for how to use naturalistic intervention strategies. 
Participants found these videos extremely useful and extremely important. At the same time, however, they were 
dissatisfied with the lengthy download times necessary to access the videos (up to 20 minutes on some of their 
Internet connections). Participants were clear that they would discontinue any download that took more than 15-30 
seconds. As a result of this conflict between the importance of video examples and time restriction on downloads, 
we have determined that for our purposes, a fully internet-based Website will be ineffective. We are currently 
working on development of a CDROM/Intemet hybrid in which videos are provided to consumers on CDROM. 
These videos are then directly accessed via hyperlinks on the Website. In this manner, the curriculum itself can be 
modified as necessary while the video examples can be instantaneously accessible regardless of curriculum changes. 
These two changes to our Website goals and design present the most significant results of our focus group. Other 
minor changes such as the use of frames, the wording of written materials, and alterations of screen space 
management have also been made based on feedback from the focus group. 

Can Effective Consumer Input Be Obtained When a Focus Group Is Conducted Over the Internet 

Based on the above results, it is our opinion that effective consumer input can be achieved through internet- 
based focus groups. In fact, there may be several advantages to conducting focus groups over the Internet. First, 
participation took less time (drive time, time spent arranging for childcare) than when sessions were face-to-face. 
Second, as researchers, all conversation was immediately transcribed at a 100% level of accuracy. Third, 
participants seemed more willing than when in person to offer critical feedback, that is, they seemed less concerned 
about offending the moderator or the Website developer when sessions occurred online. An example of this type of 
feedback can be seen in the following excerpt where one participant comments on the font used in the first prototype 
Website: “The choice of fonts on the top horizontal buttons is kind of ugly.” 

There were disadvantages, however, to the Internet-based sessions that must also be addressed. First, 
although more critical feedback was offered, it was more likely to be offered in a manner that could be considered 
inconsiderate. Because there is no opportunity to evaluate nonverbal communication, there may be more chances 
that participants will be offended and reduce their participation without the moderator’s awareness. Second, it was 
difficult at times to determine whether participants were thinking, avoiding participation, or experiencing technical 
difficulties. Again, the lack of nonverbal feedback makes it difficult for the moderator to both determine what may 
be happening and therefore make appropriate corrections. Within the areas of technical difficulties, persons with 
reading difficulties and persons who cannot touch-type can be at quite a disadvantage at the current state of internet- 
based chat rooms. Third, because participants entered the online sessions from their homes, they were more easily 
and more frequently distracted by their children. 

Many of the positives and negatives of our Internet-based focus groups can be seen as trade-offs. For 
example, participants did not have to arrange for childcare, but they were more easily distracted by their children. 
There was a greater likelihood for critical feedback, but that feedback was less likely to be provided in a considerate 
manner. Ease of participation was compromised by technical difficulties at both the human and the computer level. 
Persons wishing to employ Internet-based focus groups must, therefore, strongly consider these trade-offs and plan 
how to minimize the negative aspects of either face-to-face or Internet-based focus groups. 

Are There Differences In Focus Group Effectiveness and Interaction When Conducted in Person or Over the 
Internet 




272 e: 



There was no indication that participants provided less information in less detail over the Internet than they 
did when meeting face-to-face. It may be important to the process of using the Internet and for offering opinions in 
the focus group, of course, that participants had participated in initial face-to-face sessions. The differences in group 
effectiveness, as noted above, can be considered more as trade-offs than as inherent advantages of one form over the 
other. The feedback we received both over the Internet and face-to-face was invaluable in helping to develop a 
consumer oriented, effective web-based curriculum. 

There were differences in interactions among group members or th? basis of how the groups were 
conducted. In face-to-face sessions group members tended to engage in more informal communications than over 
the Internet. They also, however, tended to interrupt more frequently and to speak simultaneously rather than one at 
a time. Communications in general were lengthier in face-to-face sessions. It appeared that on the Internet sessions, 
participants were more likely to think their thoughts through before expressing them possibly as a result of having to 
write them. It is also possible that participants felt more comfortable pausing before answering knowing that their 
input would continue to be valid even if the conversation had moved to a different subject. Participants were more 
likely to comment on process during the Internet sessions. They would express communication or hardware 
concerns. Those who had more difficulty with the non-linear mode of chat communications (that is, several threads 
of conversations could be onscreen at the same time) expressed their difficulties freely. At no point during the face- 
to-face sessions was there discussion of process, suggesting that either such discussion was unnecessary or that 
participants were less willing to discuss communication difficulties in a traditional, “everybody should know how to 
do this” format. 

Results of evaluating both the face-to-face sessions and the Internet-based sessions suggest that focus 
groups can be equally effective in either format. There are differences, however, in the interactions that occur each 
format. An internet-based focus group has some interaction strategies that are different from those employed in 
face-to-face sessions. There were no indications in our sessions that one set of strategies is inherently more effective 
than the other. 

Discussion 

We found the Internet-based focus groups to be effective in providing appropriate, valuable input regarding 
the development of our Website. There was no evidence that a focus group cannot be conducted over the Internet. 

It was clear, however, that interactions and protocols will take a slightly different form on the Internet. Moderators 
who rigidly apply face-to-face conventions will find themselves struggling in an online environment. Moderators 
who recognize the broader goal of focus groups — organized but informal group discussion that addresses a 
particular topic or topics (Vaughn, et al., 1996) — will find themselves capable of developing and using strategies 
that facilitate the group in whatever medium the sessions occur. 

In light of the differences we discovered between face-to-face and Internet-based focus groups, we would 
offer the following suggestions to researchers who consider using Internet-based focus groups. First, have at least 
one face-to-face session prior to meeting online. Although, especially in rural environments, face-to-face meetings 
are more difficult to arrange, they allow for participants to meet in a medium they are comfortable with, to develop 
relationships in that medium, and to begin the process of group formation. Also, it is easier for the moderator to 
establish an appropriate relationship with group members. Second, have a practice chat-room session in which little 
if any issues of substance are addressed (it is possible that in a facility with a “computer lab” this practice session 
could occur in conjunction with the face-to-face session). This session should focus on technological issues, on 
developing mutually agreeable protocols, and on helping participants become familiar with participation in a chat- 
room. Third, prepare participants for possible problems such as disconnections and slow reaction times from other 
participants. Fourth, set time limits for responses to each question. Agreeing ahead of time that each question will 
be “on the table” for perhaps five minutes or ten minutes allows the moderator to introduce new questions or 
discussion threads without fear of cutting off dialogue. Fifth, be flexible. Internet-based communications are in 
their infancy. We are all learning new methods of communication and of expressing ourselves at a distance. 
Recognize that old patterns of communication may not apply and be ready to attempt new strategies on a case-by- 
case basis. 

In conclusion, our program, The SPIES Outreach Project used a combination of face-to-face and Internet- 
based focus groups to gain consumer input into the Website we were building. The logistical difficulties of 
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conducting face-to-face focus groups in our rural setting were overcome by our willingness to consider and evaluate 
an Internet-based protocol. We recommend that other programs with similar needs and with similar rural issues 
consider the Internet-based focus group as a viable research option. We hope that as research in this area grows, as 
the body of knowledge about effective online communication and relationship development grows, a better 
understanding of the qualities of effective Internet-based focus groups will ensue to the benefit of service providers 
and consumers alike. 
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